Ghost Image Mitigation in See-Through Displays With Pixel Arrays

ABSTRACT

A head-mounted apparatus include an eyepiece that include a variable dimming assembly and a frame mounting the eyepiece so that a user side of the eyepiece faces a towards a user and a world side of the eyepiece opposite the first side faces away from the user. The dynamic dimming assembly selectively modulates an intensity of light transmitted parallel to an optical axis from the world side to the user side during operation. The dynamic dimming assembly includes a variable birefringence cell having multiple pixels each having an independently variable birefringence, a first linear polarizer arranged on the user side of the variable birefringence cell, the first linear polarizer being configured to transmit light propagating parallel to the optical axis linearly polarized along a pass axis of the first linear polarizer orthogonal to the optical axis, a quarter wave plate arranged between the variable birefringence cell and the first linear polarizer, a fast axis of the quarter wave plate being arranged relative to the pass axis of the first linear polarizer to transform linearly polarized light transmitted by the first linear polarizer into circularly polarized light, and a second linear polarizer on the world side of the variable birefringence cell.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Provisional Application No.62/887,639, entitled GHOST IMAGE MITIGATION IN SEE-THROUGH DISPLAYS WITHPIXEL ARRAYS, filed on Aug. 15, 2019, the entire contents of which isincorporated by reference.

INCORPORATION BY REFERENCE

This application incorporates by reference the entirety of each of thefollowing patent applications: U.S. patent application Ser. No.15/479,700, filed on Apr. 5, 2017, published on Oct. 12, 2017 as U.S.Publication No. 2017/0293141; U.S. patent application Ser. No.16/214,575, filed on Dec. 10, 2018, published on Jun. 13, 2019 as U.S.Publication No. 2019/0179057; U.S. Provisional Patent Application Ser.No. 62/725,993, entitled SPATIALLY-RESOLVED DYNAMIC DIMMING FORAUGMENTED REALITY DEVICE, filed on Aug. 31, 2018; U.S. ProvisionalPatent Application Ser. No. 62/731,755, entitled SYSTEMS AND METHODS FOREXTERNAL LIGHT MANAGEMENT, filed on Sep. 14, 2018; U.S. ProvisionalPatent Application Ser. No. 62/858,252, entitled SPATIALLY-RESOLVEDDYNAMIC DIMMING FOR AUGMENTED REALITY DEVICE, filed on Jun. 6, 2019; andU.S. Provisional Patent Application Ser. No. 62/870,896, entitledGEOMETRIES FOR MITIGATING ARTIFACTS IN SEE-THROUGH PIXEL ARRAYS, filedon Jul. 5, 2019. For every document incorporated herein, in case ofconflict, the current specification controls.

BACKGROUND OF THE INVENTION

Modern computing and display technologies have facilitated thedevelopment of systems for so called “virtual reality” or “augmentedreality” experiences, wherein digitally reproduced images or portionsthereof are presented to a user in a manner wherein they seem to be, ormay be perceived as, real. A virtual reality, or “VR,” scenariotypically involves presentation of digital or virtual image informationwithout transparency to other actual real-world visual input; anaugmented reality, or “AR,” scenario typically involves presentation ofdigital or virtual image information as an augmentation to visualizationof the actual world around the user.

Despite the progress made in these display technologies, there is a needin the art for improved methods, systems, and devices related toaugmented reality systems, particularly, display systems.

SUMMARY OF THE INVENTION

Segmented attenuation using a polarized TFT-LCD panel can greatlyincrease the visibility and solidity of content without dimming the fullfield of view of the world. The display light in diffractive waveguidetype see through AR displays send light in two directions: one towardsthe user and one towards the world. The back reflection of the light(e.g., off of reflective components of the dimming assembly, such asmetal traces, conductors, layer index mismatches, and TFTs) going towardthe world may appear as a “ghost” image next to the primary displayimage. This ghost image impacts the effective contrast and viewingfidelity of the virtual content and therefore quality of immersion.Mitigating ghost images and stray light paths due to the metal traces,TFTs, layer index mismatches, elements/objects beyond the dimmer, etc.is a difficult problem.

The systems and techniques disclosed herein leverage polarization filmsof the dimmer as both a system optical isolator and an intra-dimmeroptical isolator to effectively suppress such “ghost” images. Theaddition of index matching fluid/gel between the eyepiece cover glassand the dimmer, where the index is close to the first layer of thedimmer, can mitigate ghosting from the first dimmer surface.

Using quarter waveplates (QWPs) with specifically chosen achromaticproperties can allow a liquid crystal dimmer, such as those using anelectrically controlled birefringence (ECB) cell, to have lesspolarization leakage, better chromatic performance and be more colorneutral across a relatively wide range of operating conditions.

The present disclosure relates generally to techniques for improvingoptical systems in varying ambient light conditions. More particularly,embodiments of the present disclosure provide systems and methods foroperating an augmented reality (AR) device comprising a dimming element.Although the present invention is described in reference to an ARdevice, the disclosure is applicable to a variety of applications incomputer vision and image display systems.

In general, in a first aspect, the invention features head-mountedapparatus that include an eyepiece that include a variable dimmingassembly and a frame mounting the eyepiece so that, during use of thehead-mounted apparatus, a user side of the eyepiece faces a towards auser of the head-mounted apparatus, and a world side of the eyepieceopposite the first side faces away from the user. The dynamic dimmingassembly is configured to selectively modulate an intensity of lighttransmitted parallel to an optical axis from the world side of theeyepiece to the user side of the eyepiece during operation of thehead-mounted apparatus. The dynamic dimming assembly includes a variablebirefringence cell having multiple pixels each having an independentlyvariable birefringence, a first linear polarizer arranged on the userside of the variable birefringence cell, the first linear polarizerbeing configured to transmit light propagating parallel to the opticalaxis linearly polarized along a pass axis of the first linear polarizerorthogonal to the optical axis, a quarter wave plate arranged betweenthe variable birefringence cell and the first linear polarizer, a fastaxis of the quarter wave plate being arranged relative to the pass axisof the first linear polarizer to transform linearly polarized lighttransmitted by the first linear polarizer into circularly polarizedlight, and a second linear polarizer on the world side of the variablebirefringence cell.

Implementations of the head-mounted apparatus can include one or more ofthe following features and/or features of other aspects. For example,the dynamic dimming assembly further includes an optical retarderarranged between the variable birefringence cell and the second linearpolarizer. The optical retarder can be a second quarter wave plate. Theoptical retarder is an A-plate with a retardation greater than aretardation of the quarter wave plate. A difference between aretardation of the optical retarder and a retardation of the quarterwave plate can correspond to a residual retardation of the variablebirefringent cell in a minimum birefringence state.

The variable birefringence cell can include a layer of a liquid crystal.The liquid crystal can be configured in an electrically controllablebirefringence mode. In some embodiments, the layer of the liquid crystalis a vertically aligned liquid crystal layer. The liquid crystal can bea nematic phase liquid crystal.

The pixels of the variable birefringence cell can be actively addressedpixels.

The eyepiece can further include a see-through display mounted in theframe on the user side of the variable dimming assembly. The see-throughdisplay can include one or more waveguide layers arranged to receivelight from a light projector during operation of the head-mountedapparatus and direct the light toward the user. The head-mountedapparatus can include one or more index-matching layers arranged betweenthe see-through display and the variable dimming assembly.

In some embodiments, the dynamic dimming assembly includes one or moreantireflection layers.

In another aspect, the invention features an augmented reality systemincluding the head-mounted apparatus.

In general, in another aspect, the invention features head-mountedapparatus that include an eyepiece having a variable dimming assemblyand a frame mounting the eyepiece so that, during use of thehead-mounted apparatus, a user side of the eyepiece faces a towards auser of the head-mounted apparatus, and a world side of the eyepieceopposite the first side faces away from the user. The dynamic dimmingassembly is configured to selectively modulate an intensity of lighttransmitted parallel to an optical axis from the world side of theeyepiece to the user side of the eyepiece during operation of thehead-mounted apparatus. The dynamic dimming assembly includes a layer ofa liquid crystal, a circular polarizer arranged on the user side of theliquid crystal layer, and a linear polarizer on the world side of theliquid crystal layer.

Embodiments of the head-mounted apparatus can include one or more of thefollowing features and/or features of other aspects. For example, thecircular polarizer can include a linear polarizer and a quarter waveplate. The head-mounted apparatus can include an A-plate arrangedbetween the linear polarizer on the world side of the liquid crystallayer.

The head-mounted apparatus can include a pixelated cell including thelayer of the liquid crystal, the pixelated cell being an activelyaddressed pixelated cell.

In general, in a further aspect, the invention features a method ofoperating an optical system. The method may include receiving, at theoptical system, light associated with a world object. The method mayalso include projecting a virtual image onto an eyepiece. The method mayfurther include determining a portion of a system field of view of theoptical system to be at least partially dimmed based on detectedinformation. The method may further include adjusting a dimmer to reducean intensity of the light associated with the world object in theportion of the system field of view.

In some embodiments, the optical system includes a light sensorconfigured to detect light information corresponding to the lightassociated with the world object. In some embodiments, the detectedinformation includes the light information. In some embodiments, thelight information includes multiple spatially-resolved light values. Insome embodiments, the light information includes a global light value.In some embodiments, the optical system includes an eye trackerconfigured to detect gaze information corresponding to an eye of a userof the optical system. In some embodiments, the detected informationincludes the gaze information. In some embodiments, the gaze informationincludes a pixel location that intersects with a gaze vector of the eyeof the user. In some embodiments, the gaze information includes one ormore of a pupil position of the eye of the user, a center of rotation ofthe eye of the user, a pupil size of the eye of the user, a pupildiameter of the eye of the user, and cone and rod locations of the eyeof the user. In some embodiments, the method further includes detectingimage information corresponding to the virtual image. In someembodiments, the detected information includes the image information. Insome embodiments, the image information includes a plurality ofspatially-resolved image brightness values. In some embodiments, theimage information includes a global image brightness value.

In some embodiments, the method further includes determining multiplespatially-resolved dimming values for the portion of the system field ofview based on the detected information. In some embodiments, the dimmeris adjusted according to the plurality of dimming values. In someembodiments, the dimmer comprises a plurality of pixels. In someembodiments, the dimmer is adjusted to completely block the intensity ofthe light associated with the world object in all of the system field ofview. In some embodiments, the method further includes adjusting abrightness associated with the virtual image. In some embodiments, thevirtual image is characterized by an image field of view. In someembodiments, the image field of view is equal to the system field ofview. In some embodiments, the image field of view is a subset of thesystem field of view.

In general, in another aspect, the invention features an optical system.The optical system may include a projector configured to project avirtual image onto an eyepiece. The optical system may also include adimmer configured to dim light associated with a world object. Theoptical system may further include a processor communicatively coupledto the projector and the dimmer. In some embodiments, the processor isconfigured to perform operations including determining a portion of asystem field of view of the optical system to be at least partiallydimmed based on detected information. In some embodiments, theoperations may also include adjusting the dimmer to reduce an intensityof the light associated with the world object in the portion of thesystem field of view.

In some embodiments, the optical system further includes a light sensorconfigured to detect light information corresponding to the lightassociated with the world object. In some embodiments, the detectedinformation includes the light information. In some embodiments, thelight information includes a plurality of spatially-resolved lightvalues. In some embodiments, the light information includes a globallight value. In some embodiments, the optical system further includes aneye tracker configured to detect gaze information corresponding to aneye of a user of the optical system. In some embodiments, the detectedinformation includes the gaze information. In some embodiments, the gazeinformation includes a pixel location that intersects with a gaze vectorof the eye of the user. In some embodiments, the gaze informationincludes one or more of a pupil position of the eye of the user, acenter of rotation of the eye of the user, a pupil size of the eye ofthe user, a pupil diameter of the eye of the user, and cone and rodlocations of the eye of the user. In some embodiments, the operationsfurther include detecting image information corresponding to the virtualimage. In some embodiments, the detected information includes the imageinformation. In some embodiments, the image information includes aplurality of spatially-resolved image brightness values. In someembodiments, the image information includes a global image brightnessvalue.

In some embodiments, the operations further include determining multiplespatially-resolved dimming values for the portion of the system field ofview based on the detected information. In some embodiments, the dimmeris adjusted according to the plurality of dimming values. In someembodiments, the dimmer comprises a plurality of pixels. In someembodiments, the dimmer is adjusted to completely block the intensity ofthe light associated with the world object in all of the system field ofview. In some embodiments, the operations further include adjusting abrightness associated with the virtual image. In some embodiments, thevirtual image is characterized by an image field of view. In someembodiments, the image field of view is equal to the system field ofview. In some embodiments, the image field of view is a subset of thesystem field of view.

Numerous benefits can be achieved by way of the present disclosure overconventional techniques. For example, the AR device described herein maybe used in varying light levels, from dark indoors to bright outdoors,by globally dimming and/or selectively dimming the ambient lightreaching the user's eyes. Embodiments of the present invention allow forAR and virtual reality (VR) capabilities in a single device by using thepixelated dimmer to attenuate the world light by greater than 99%.Embodiments of the present invention also mitigate vergenceaccommodation conflict using a variable focal element with discrete orcontinuous variable depth plane switching technologies. Embodiments ofthe present invention improve the battery life of the AR device byoptimizing the projector brightness based on the amount of detectedambient light. Other benefits will be readily apparent to those skilledin the art.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an augmented reality (AR) scene as viewed through awearable AR device according to some embodiments described herein.

FIG. 2A illustrates one or more general features of an AR deviceaccording to the present invention.

FIG. 2B illustrates an example of an AR device in which a dimmed area isdetermined based on detected light information.

FIG. 2C illustrates an example of an AR device in which a dimmed area isdetermined based on a virtual image.

FIG. 2D illustrates an example of an AR device in which a dimmed area isdetermined based on gaze information.

FIG. 3 illustrates a schematic view of a wearable AR device according tothe present invention.

FIG. 4 illustrates a method for operating an optical system.

FIG. 5 illustrates an AR device with an eyepiece and a pixelated dimmingelement.

FIG. 6A is a front view of an example optically-transmissive spatiallight modulator (“SLM”) or display assembly for a see-through displaysystem, according to some embodiments of the present disclosure.

FIG. 6B is a schematic side view of the example SLM or display assemblyof FIG. 6A.

FIG. 7A illustrates an example of a see-through display system in afirst state.

FIG. 7B illustrates an example of the see-through display system of FIG.7A in a second state different from the first state.

FIG. 8A illustrates an example of another see-through display system ina first state.

FIG. 8B illustrates an example of the see-through display system of FIG.8A in a second state different from the first state.

FIG. 9 illustrates an example of a see-through display system accordingto some embodiments of the present disclosure.

FIG. 10 illustrates an example of a see-through display system accordingto other embodiments of the present disclosure.

FIG. 11 illustrates an example of a see-through display system accordingto yet other embodiments of the present disclosure.

FIG. 12 is a diagram of an example computer system useful with asee-through display system.

DETAILED DESCRIPTION

An ongoing technical challenge with optical see through (OST) augmentedreality (AR) devices is the variation in the opacity and/or visibilityof the virtual content under varying ambient light conditions. Theproblem worsens in extreme lighting conditions such as a completely darkroom or outside in full bright sunlight. Embodiments disclosed hereincan reduce (e.g., solve) these and other problems by dimming the worldlight at different spatial locations within the field of view of an ARdevice. In such variable dimming arrangements, the AR device candetermine which a portion of the field of view to dim and the amount ofdimming that is applied each portion based on various informationdetected by the AR device. This information may include detected ambientlight, detected gaze information, and/or the detected brightness of thevirtual content being projected. The functionality of the AR device canbe further improved by detecting a direction associated with the ambientlight by, for example, detecting spatially-resolved light intensityvalues. This can improve the AR device's battery life by dimming onlythose portions of the field of view in which dimming is needed and/orincreasing the projector brightness in certain portions of the field ofview. Accordingly, embodiments disclosed herein can enable usage of theAR device in a much wider variety of ambient lighting conditions thantraditionally possible.

FIG. 1 illustrates an AR scene as viewed through a wearable AR deviceaccording to some embodiments described herein. An AR scene 100 isdepicted wherein a user of an AR technology sees a real-world park-likesetting 106 featuring people, trees, buildings in the background, and aconcrete platform 120. In addition to these items, the user of the ARtechnology also perceives that he “sees” a robot statue 110 standingupon the real-world platform 120, and a cartoon-like avatar character102 flying by, which seems to be a personification of a bumble bee, eventhough these elements (character 102 and statue 110) do not exist in thereal world. Due to the extreme complexity of the human visual perceptionand nervous system, it is challenging to produce a virtual reality (VR)or AR technology that facilitates a comfortable, natural-feeling, richpresentation of virtual image elements amongst other virtual orreal-world imagery elements.

FIG. 2A illustrates one or more general features of an example AR device200. AR device 200 includes an eyepiece 202 and a dynamic dimmer 203configured to be transparent or semi-transparent when AR device 200 isin an inactive mode or an off mode such that a user may view one or moreworld objects 230 when looking through eyepiece 202 and dynamic dimmer203. As illustrated, eyepiece 202 and dynamic dimmer 203 are arranged ina side-by-side configuration and form a system field of view that a usersees when looking through eyepiece 202 and dynamic dimmer 203. In someembodiments, the system field of view is defined as the entiretwo-dimensional region occupied by one or both of eyepiece 202 anddynamic dimmer 203. Although FIG. 2A illustrates a single eyepiece 202and a single dynamic dimmer 203 (for illustrative reasons), in generalAR device 200 includes two eyepieces and two dynamic dimmers, one foreach eye of a user.

During operation, dynamic dimmer 203 may be adjusted to vary anintensity of a world light 232 from world objects 230 transmitted toeyepiece 202 and the user, thereby providing a dimmed area 236 withinthe system field of view, which transmits less world light that theother areas of dynamic dimmer 203. Dimmed area 236 may be a portion orsubset of the system field of view, and may be partially or completelydimmed. A partially dimmed area will transmit a fraction of incidentworld light, while a completely dimmed area will block substantially allincident world light. Dynamic dimmer 203 may be adjusted according to aplurality of spatially-resolved dimming values for dimmed area 236.

Furthermore, during operation of AR device 200, projector 214 mayproject a virtual image 222 onto eyepiece 202 which may be observed bythe user along with world light 232. Projecting virtual image 222 ontoeyepiece 202 projects a light field 223 (i.e., an angular representationof virtual content) onto the user's retina so that the user perceivesthe corresponding virtual content as being positioned at some locationwithin the user's environment. It should be noted that the virtualcontent (character 102 and statue 110) is depicted in FIGS. 2A-2D asbeing displayed at eyepiece 202 for illustrative purposes only. Thevirtual content may actually be perceived by the user at various depthsbeyond eyepiece 202. For example, the user may perceive statue 110 asbeing located at approximately the same distance as world objects 230(i.e., platform 120) and character 102 as being located closer to theuser. In some embodiments, dynamic dimmer 203 may be positioned closerto the user than eyepiece 202 and may accordingly reduce an intensity ofthe light associated with virtual image 222 (i.e., light field 223). Insome embodiments, two dynamic dimmers may be utilized, one on each sideof eyepiece 202.

As depicted, AR device 200 includes an ambient light sensor 234configured to detect world light 232. Ambient light sensor 234 may bepositioned such that world light 232 detected by ambient light sensor234 is similar to and/or representative of world light 232 that impingeson dynamic dimmer 203 and/or eyepiece 202. In some embodiments, ambientlight sensor 234 may be configured to detect a plurality ofspatially-resolved light values corresponding to different pixels ofdynamic dimmer 203. In some embodiments, or in the same embodiments,ambient light sensor 234 may be configured to detect a global lightvalue corresponding to an average light intensity or a single lightintensity of world light 232. Other possibilities are contemplated.

FIG. 2B illustrates AR device 200 in a state in which dimmed area 236 isdetermined based on detected light information corresponding to worldlight 232. Specifically, ambient light sensor 234, in this example,detects world light 232 from the sun 233 and may further detect adirection and/or a portion of the system field of view at which worldlight 232 associated with the sun passes through AR device 200. Inresponse, dynamic dimmer 203 is adjusted to set dimmed area 236 to covera portion of the system field of view corresponding to the detectedworld light, reducing an intensity of world light from sun 233 toeyepiece 202 and the user. As illustrated, dynamic dimmer 203 isadjusted to reduce the transmitted intensity of world light 232 at thecenter of dimmed area 236 at a greater amount than the extremities ofdimmed area 236.

FIG. 2C illustrates AR device 200 in a state in which dimmed area 236 isdetermined based on the location of virtual image 222 in the field ofview. Specifically, dimmed area 236 is determined based on the virtualcontent perceived by the user resulting from the user observing virtualimage 222. In some embodiments, AR device 200 may detect imageinformation that includes a location of virtual image 222 (e.g., alocation within the system's field of view and/or the correspondinglocation of dynamic dimmer 203 through which the user perceives thevirtual content) and/or a brightness of virtual image 222 (e.g., abrightness of the perceived virtual content), among other possibilities.As illustrated, dynamic dimmer 203 may be adjusted to set dimmed area236 to cover a portion of the system field of view corresponding tovirtual image 222 or, alternatively, in some embodiments dimmed area 236may cover a portion of the system field of view that is not aligned withvirtual image 222. In some embodiments, the dimming values of dimmedarea 236 may be determined based on world light 232 detected by ambientlight sensor 234 and/or the brightness of virtual image 222.

FIG. 2D illustrates AR device 200 in a state in which dimmed area 236 isdetermined based on gaze information corresponding to an eye of a user.In some embodiments, the gaze information includes a gaze vector 238 ofthe user and/or a pixel location of dynamic dimmer 203 at which gazevector 238 intersects with dynamic dimmer 203. As illustrated, dynamicdimmer 203 may be adjusted to set dimmed area 236 to cover a portion ofthe system field of view corresponding to an intersection point (orintersection region) between gaze vector 238 and dynamic dimmer 203 or,alternatively, in some embodiments dimmed area 236 may cover a portionof the system field of view that does not correspond to the intersectionpoint (or intersection region) between gaze vector 238 and dynamicdimmer 203. In some embodiments, the dimming values of dimmed area 236may be determined based on world light 232 detected by ambient lightsensor 234 and/or the brightness of virtual image 222. In someembodiments, gaze information may be detected by an eye tracker 240mounted to AR device 200.

FIG. 3 illustrates a schematic view of a further example wearable ARdevice 300. AR device 300 includes a left eyepiece 302A and a leftdynamic dimmer 303A arranged in a side-by-side configuration and a righteyepiece 302B and a right dynamic dimmer 303B also arranged in aside-by-side configuration. As depicted, AR device 300 includes one ormore sensors including, but not limited to: a left front-facing worldcamera 306A attached directly to or near left eyepiece 302A, a rightfront-facing world camera 306B attached directly to or near righteyepiece 302B, a left side-facing world camera 306C attached directly toor near left eyepiece 302A, a right side-facing world camera 306Dattached directly to or near right eyepiece 302B, a left eye tracker340A positioned so as to observe a left eye of a user, a right eyetracker 340B positioned so as to observe a right eye of a user, and anambient light sensor 334. AR device 300 also includes one or more imageprojection devices such as a left projector 314A optically linked toleft eyepiece 302A and a right projector 314B optically linked to righteyepiece 302B.

Some or all of the components of AR device 300 may be head mounted suchthat projected images may be viewed by a user. In some implementations,all of the components of AR device 300 shown in FIG. 3 are mounted ontoa single device (e.g., a single headset) wearable by a user. In certainimplementations, a processing module 350 is physically separate from andcommunicatively coupled to the other components of AR device 300 by oneor more wired and/or wireless connections. For example, processingmodule 350 may be mounted in a variety of configurations, such asfixedly attached to a frame, fixedly attached to a helmet or hat worn bya user, embedded in headphones, or otherwise removably attached to auser (e.g., in a backpack-style configuration, in a belt-coupling styleconfiguration, etc.).

Processing module 350 may include a processor 352 and an associateddigital memory 356, such as non-volatile memory (e.g., flash memory),both of which may be utilized to assist in the processing, caching, andstorage of data. The data may include data captured from sensors (whichmay be, e.g., operatively coupled to AR device 300) or otherwiseattached to a user, such as cameras 306, ambient light sensor 334, eyetrackers 340, microphones, inertial measurement units, accelerometers,compasses, GPS units, radio devices, and/or gyros. For example,processing module 350 may receive image(s) 320 from cameras 306.Specifically, processing module 350 may receive left front image(s) 320Afrom left front-facing world camera 306A, right front image(s) 320B fromright front-facing world camera 306B, left side image(s) 320C from leftside-facing world camera 306C, and right side image(s) 320D from rightside-facing world camera 306D. In some embodiments, image(s) 320 mayinclude a single image, a pair of images, a video comprising a stream ofimages, a video comprising a stream of paired images, and the like.Image(s) 320 may be periodically generated and sent to processing module350 while AR device 300 is powered on, or may be generated in responseto an instruction sent by processing module 350 to one or more of thecameras. As another example, processing module 350 may receive lightinformation from ambient light sensor 334. As another example,processing module 350 may receive gaze information from one or both ofeye trackers 340. As another example, processing module 350 may receiveimage information (e.g., image brightness values) from one or both ofprojectors 314.

Eyepieces 302A and 302B may include transparent or semi-transparentwaveguides configured to direct light from projectors 314A and 314B,respectively. Specifically, processing module 350 may cause leftprojector 314A to output a left virtual image 322A onto left eyepiece302A (causing a corresponding light field associated with left virtualimage 322A to be projected onto the user's retina), and may cause rightprojector 314B to output a right virtual image 322B onto right eyepiece302B (causing a corresponding light field associated with right virtualimage 322B to be projected onto the user's retina). In some embodiments,each of eyepieces 302 includes multiple waveguides corresponding todifferent colors and/or different depth planes. In some embodiments,dynamic dimmers 303 may be coupled to and/or integrated with eyepieces302. For example, one of dynamic dimmers 303 may be incorporated into amulti-layer eyepiece and may form one or more layers that make up one ofeyepieces 302.

Cameras 306A and 306B may be positioned to capture images thatsubstantially overlap with the field of view of a user's left and righteyes, respectively. Accordingly, placement of cameras 306 may be near auser's eyes but not so near as to obscure the user's field of view.Alternatively or additionally, cameras 306A and 306B may be positionedso as to align with the incoupling locations of virtual images 322A and322B, respectively. Cameras 306C and 306D may be positioned to captureimages to the side of a user, e.g., in a user's peripheral vision oroutside the user's peripheral vision. Image(s) 320C and 320D capturedusing cameras 306C and 306D need not necessarily overlap with image(s)320A and 320B captured using cameras 306A and 306B.

One or more components of AR device 300 may be similar to one or morecomponents described in reference to FIGS. 2A-2D. For example, in thesome embodiments the functionality of eyepieces 302, dynamic dimmers303, projectors 314, ambient light sensor 334, and eye trackers 340 maybe similar to eyepiece 202, dynamic dimmer 203, projector 214, ambientlight sensor 234, and eye tracker 240, respectively. In someembodiments, the functionality of processing module 350 may beimplemented by two or more sets of electronic hardware components thatare housed separately but communicatively coupled. For example, thefunctionality of processing module 350 may be carried out by electronichardware components housed within a headset in conjunction withelectronic hardware components housed within a computing devicephysically tethered to the headset, one or more electronic deviceswithin the environment of the headset (e.g., smart phones, computers,peripheral devices, smart appliances, etc.), one or moreremotely-located computing devices (e.g., servers, cloud computingdevices, etc.), or a combination thereof. One example of such aconfiguration is described in further detail below with reference toFIG. 12.

FIG. 4 illustrates an example method 400 for operating an optical system(e.g., AR device 200 or 300). Generally, operation of optical systemscan include performing the steps of method 400 as shown in FIG. 4, or ina different order and not all of the steps need be performed. Forexample, in some embodiments, one or more of steps 406, 408, and 410 maybe omitted during performance of method 400. One or more steps of method400 may be performed by a processor (e.g., processor 352) or by someother component within the optical system.

At step 402, light (e.g., world light 232) associated with an object(e.g., world object 230) is received at the optical system. The objectmay be a real-world object, such as a tree, a person, a house, abuilding, the sun, etc., that is viewed by a user of the optical system.In some embodiments, the light associated with the object is firstreceived by a dynamic dimmer (e.g., dynamic dimmer 203 or 303) or by anexternal cosmetic lens of the optical system. In some embodiments, thelight associated with the object is considered to be received at theoptical system when the light reaches one or more components of theoptical system (e.g., when the light reaches the dynamic dimmer).

At step 404, a virtual image (e.g., virtual image 222 or 322) isprojected onto an eyepiece (e.g., eyepiece 202 or 302). The virtualimage may be projected onto the eyepiece by a projector (e.g., projector214 or 314) of the optical system. The virtual image may be a singleimage, a pair of images, a video composed of a stream of images, a videocomposed of a stream of paired images, and the like. In someembodiments, the virtual image is considered to be projected onto theeyepiece when any light associated with the virtual image reaches theeyepiece. In some embodiments, projecting the virtual image onto theeyepiece causes a light field (i.e., an angular representation ofvirtual content) to be projected onto the user's retina in a manner suchthat the user perceives the corresponding virtual content as beingpositioned at some location within the user's environment.

During steps 406, 408, and 410, information may be detected by theoptical system using, for example, one or more sensors of the opticalsystem. At step 406, light information corresponding to the lightassociated with the object is detected. The light information may bedetected using a light sensor (e.g., ambient light sensor 234 or 334)mounted to the optical system. In some embodiments, the lightinformation includes a plurality of spatially-resolved light values.Each of the plurality of spatially-resolved light values may correspondto a two-dimensional position within the system field of view. Forexample, each of the light values may be associated with a pixel of thedynamic dimmer. In other embodiments, or in the same embodiments, thelight information may include a global light value. The global lightvalue may be associated with the entire system field of view (e.g., anaverage light value of light impinging on all pixels of the dynamicdimmer).

At step 408, gaze information corresponding to an eye of a user of theoptical system is detected. The gaze information may be detected usingan eye tracker (e.g., eye tracker 240 or 340) mounted to the opticalsystem. In some embodiments, the gaze information includes a gaze vector(e.g., gaze vector 238) of the eye of the user. In some embodiments, thegaze information includes one or more of a pupil position of the eye ofthe user, a center of rotation of the eye of the user, a pupil size ofthe eye of the user, a pupil diameter of the eye of the user, and coneand rod locations of the eye of the user. The gaze vector may bedetermined based on one or more components of the gaze information, suchas the pupil position, the center of rotation of the eye, the pupilsize, the pupil diameter, and/or the cone and rod locations. When thegaze vector is determined based on the cone and rod locations, it mayfurther be determined based on the light information (e.g., the globallight value) so as to determine an origin of the gaze vector within aretinal layer of the eye containing the cone and rod locations. In someembodiments, the gaze information includes a pixel or group of pixels ofthe dynamic dimmer at which the gaze vector intersects with the dynamicdimmer.

At step 410, image information corresponding to a virtual image (e.g.,virtual image 222 or 322) projected by the projector onto the eyepieceis detected. The image information may be detected by the projector, bya processor (e.g., processor 352), or by a separate light sensor. Insome embodiments, the image information includes one or more locationswithin the dynamic dimmer through which the user perceives the virtualcontent when the user observes the virtual image. In some embodiments,the image information includes a plurality of spatially-resolved imagebrightness values (e.g., brightness of the perceived virtual content).For example, each of the image brightness values may be associated witha pixel of the eyepiece or of the dynamic dimmer. In one particularimplementation, when the processor sends instructions to the projectorto project the virtual image onto the eyepiece, the processor maydetermine, based on the instructions, the spatially-resolved imagebrightness values. In another particular implementation, when theprojector receives the instructions from the processor to project thevirtual image onto the eyepiece, the projector sends thespatially-resolved image brightness values to the processor. In anotherparticular implementation, a light sensor positioned on or near theeyepiece detects and sends the spatially-resolved image brightnessvalues to the processor. In other embodiments, or in the sameembodiments, the image information includes a global image brightnessvalue. The global image brightness value may be associated with theentire system field of view (e.g., an average image brightness value ofthe entire virtual image).

At step 412, a portion of the system field of view to be at leastpartially dimmed is determined based on the detected information. Thedetected information may include the light information detected duringstep 406, the gaze information detected during step 408, and/or theimage information detected during step 410. In some embodiments, theportion of the system field of view is equal to the entire system fieldof view. In various embodiments, the portion of the system field of viewmay be equal to 1%, 5%, 10%, 25%, 50%, or 75%, etc., of the system fieldof view. In some embodiments, the different types of information may beweighted differently in determining the portion to be at least partiallydimmed. For example, gaze information, when available, may be weightedmore heavily in determining the portion to be at least partially dimmedthan light information and image information. In one particularimplementation, each type of information may independently be used todetermine a different portion of the system field of view to be at leastpartially dimmed, and subsequently the different portions may becombined into a single portion using an AND or an OR operation.

At step 414, multiple spatially-resolved dimming values for the portionof the system field of view are determined based on the detectedinformation. In some embodiments, the dimming values are determinedusing a formulaic approach based on a desired opacity or visibility ofthe virtual content. In one particular implementation, the visibility ofthe virtual content may be calculated using the following equation:

$V = \frac{I_{\max}\left( {1 - \frac{1}{C}} \right)}{{I_{\max}\left( {1 + \frac{1}{C}} \right)} + {2I_{back}}}$

where V is the visibility, I_(max) is the brightness of the virtualimage as indicated by the image information, I_(back) is related to alight value associated with the world object as indicated by the lightinformation (which may be modified by the determined dimming value), andC is a desired contrast (e.g., 100:1). For example, the visibilityequation may be used at each pixel location of the dimmer to calculate adimming value for the particular pixel location using the brightness ofthe virtual image at the particular pixel location and the light valueassociated with the world object at the particular pixel location.

At step 416, the dimmer is adjusted to reduce an intensity of the lightassociated with the object in the portion of the system field of view.For example, the dimmer may be adjusted such that the intensity of thelight associated with the object impinging on each pixel location of thedimmer is reduced according to the dimming value determined for thatparticular pixel location. As used in the present disclosure, adjustingthe dimmer may include initializing the dimmer, activating the dimmer,powering on the dimmer, modifying or changing a previously initialized,activated, and/or powered on dimmer, and the like. In some embodiments,the processor may send data to the dimmer indicating both the portion ofthe system field of view and the plurality of spatially-resolved dimmingvalues.

At step 418, the projector is adjusted to adjust a brightness associatedwith the virtual image. For example, in some embodiments it may bedifficult to achieve a desired opacity or visibility of the virtualcontent without increasing or decreasing the brightness of the virtualobject. In such embodiments, the brightness of the virtual image may beadjusted before, after, simultaneously, or concurrently with adjustingthe dimmer.

FIG. 5 illustrates an AR device 500 with an eyepiece 502 and a pixelateddimming element 503 composed of a spatial grid of dimming areas (i.e.,pixels) that can have various levels of dimming independent of eachother (i.e., they are independently variable). Each of the dimming areasmay have an associated size (e.g., width and height) and an associatedspacing (e.g., pitch). As illustrated, the spatial grid of dimming areasmay include one or more dark pixels 506 providing complete dimming ofincident light and one or more clear pixels 508 providing completetransmission of incident light. Adjacent pixels within pixelated dimmingelement 503 may be bordering (e.g., when the pitch is equal to thewidth) or may be separated by gaps or channels (e.g., when the pitch isgreater than the width).

In various embodiments, pixelated dimming element 503 uses liquidcrystal technology. Such technology typically includes a layer of aliquid crystal material (e.g., having a nematic phase) aligned relativeto one or more electrode layers so that the liquid crystal material canbe reoriented depending on an electric field strength applied to thepixel (e.g., by applying a potential difference across the liquidcrystal layer using electrodes on opposing sides of the liquid crystallayer). Examples of liquid crystal modes include twisted nematic (“TN”)or vertically aligned (“VA”) liquid crystals. Electrically controlledbirefringence (“ECB”) liquid crystal modes can also be used. Liquidcrystal phases other than nematic phases can be used, such as,ferroelectric liquid crystals. In some embodiments, dye doped orguest-host liquid crystal materials can be used.

FIG. 6A depicts a front view of an example optically-transmissivespatial light modulator (“SLM”) or display assembly 603 for asee-through display system. Similarly, FIG. 6B depicts a schematiccross-sectional view of the example SLM or display assembly of FIG. 6A.In some examples, the optically-transmissive SLM or display assembly 603may form all or part of an external cover of an augmented realitysystem. The assembly 603 may, for example, correspond to anoptically-transmissive controllable dimming assembly that is similar orequivalent to one or more of the dimming assemblies described herein, anoptically-transmissive LCD, an optically-transmissive OLED display, andthe like. In some implementations, the assembly 603 of FIGS. 6A-6B maycorrespond to one or more of components 203, 303 a, 303B, and 503 asdescribed above with reference to FIGS. 2A-2C, 3, and 5, respectively.Additional examples of controllable dimming assembly architectures andcontrol schemes are described in further detail in U.S. ProvisionalPatent Application Ser. No. 62/725,993, U.S. Provisional PatentApplication Ser. No. 62/731,755, U.S. Provisional Patent ApplicationSer. No. 62/858,252, and U.S. Provisional Patent Application Ser. No.62/870,896, all of which are incorporated herein by reference in theirentirety.

In the example of FIGS. 6A-6B, the assembly 603 includes a liquidcrystal layer 618 sandwiched between an outer electrode layer 616A andan inner electrode layer 616B, which are in turn sandwiched between anouter polarizer 612A and an inner polarizer 612B. The outer and innerpolarizers 612A, 612B may be configured to linearly polarize unpolarizedlight that passes therethrough. The assembly 603 includes an outersubstrate layer 620A positioned between the outer polarizer 612A and theouter electrode layer 616A, an inner substrate layer 620B positionedbetween the inner polarizer 612B and the inner electrode layer 616B.Substrate layers 620A and 620B support electrode layers 616A and 616Band are typically formed from an optically transparent material, such asa glass or plastic. The assembly 603 further includes an outer opticalretarder 614A (e.g., an A-plate) positioned between the outer polarizer612A and the outer electrode layer 616A, an inner optical retarder 614B(e.g., an A-plate) positioned between the inner polarizer 612B and theinner electrode layer 616B.

As mentioned above, in some implementations, the assembly 803 mayinclude or correspond to an ECB cell. Advantageously, ECB cells can beconfigured to modulate circularly polarized light, e.g., to change theellipticity of the circularly polarized light, for example.

In operation, the outer polarizer 612A imparts a first polarizationstate (e.g., linear polarization along the vertical direction of FIG.6A) to ambient light propagating therethrough toward a user's eye. Next,depending on their orientation, liquid crystal molecules containedwithin the liquid crystal layer 618 further rotate/retard the polarizedambient light as it traverses the liquid crystal layer 618. For example,liquid crystal layer 618 can rotate the linearly polarized light so thatthe plane of polarization is different from the plane of polarization ofthe first polarization state. Alternatively, or additionally, the liquidcrystal layer can retard the polarization, e.g., transforming linearlypolarized light into elliptically or circularly polarized light.Generally, the amount of rotation/retardation depends on thebirefringence of the liquid crystal material, its orientation, and thethickness of liquid crystal layer 618. The amount ofrotation/retardation also depends on the electric field applied toliquid crystal layer 618, e.g., by applying a potential differenceacross outer and inner electrode layers 616A, 616B. It follows that theamount of polarization rotation imparted by the pair of electrode layers616A, 616B and liquid crystal layer 618 may be varied on apixel-by-pixel basis according to the voltage applied to the electrodelayers at each respective pixels.

Retardation of the polarized light is also influenced by the outer andinner optical retarders 614A, 614B. For example, use of a quarter waveplate as outer optical retarder 614A will serve to retard the linearlypolarized light transmitted by polarizer 612A to transform the linearlypolarized light to circularly polarized light where the fast axis of thequarter wave plate is appropriately oriented with respect to the passaxis of the linear polarizer (e.g., at 45°).

Lastly, the inner polarizer 612B may transmit light of a second,different polarization state (e.g., horizontal polarization) compared toouter polarizer 612A. The second polarization state may be orthogonal tothe polarization state imparted on the ambient light by outer polarizer612A. In such circumstances, where the cumulative effect of the liquidcrystal layer 618, and the outer and/or inner optical retarders 614A,614B rotates polarized light transmitted by outer polarizer 612A, innerpolarizer 612B will transmit the light transmitted by outer polarizer612A, albeit rotated by 90 degrees. Alternatively, where the cumulativeeffect of liquid crystal layer 618 and the optical retarders 614A and614B leaves the polarization state of light from polarizer 612Aunchanged, it will be blocked by inner polarizer 612B. Accordingly, theinner polarizer 612B may allow portions of ambient light in the secondpolarization state to pass therethrough unaffected, and may attenuateportions of ambient light in polarization states other than the secondpolarization state. The amount of polarization rotation can becontrolled on a pixel-by-pixel basis by the electric field strengthapplied to the liquid crystal layer at each pixel, allowing lighttransmission of device 603 to be controlled on a pixel-by-pixel basis.

Generally, the pixel structure of the electrode layers can varydepending, e.g., on the nature of the liquid crystal layer, the pixelsize, etc. In some embodiments, one of the outer electrode layer 616Aand the inner electrode layer 616B may correspond to a layer ofindividually-addressable electrodes (i.e., pixels) arranged in atwo-dimensional array. For instance, in some examples, the innerelectrode layer 616B may correspond to an array of pixel electrodes thatmay each be selectively controlled by the assembly 603 to generate arespective electric field/voltage in tandem with the outer electrodelayer 616A, which may correspond to a single planar electrode. In someexamples, the electrodes of one or both of the outer and inner electrodelayers 616A, 616B may be made out of an optically-transmissiveconducting material, such as indium tin oxide (“ITO”).

As shown in FIGS. 6A-6B, the assembly 603 also includes metal linetraces or conductors 617 a-n. Each pixel electrode in the array of pixelelectrodes of the assembly 603 is electrically coupled to acorresponding thin film transistor (“TFT”), which in turn iselectrically coupled to a corresponding pair of metal line traces orconductors in the plurality of metal line traces or conductors 617 a-n.Such metal line traces or conductors are positioned in “transmissivegap” regions between pixels (e.g., pixel electrodes of the innerelectrode layer 616B), and are further electrically coupled to one ormore circuits for driving or otherwise controlling the state of eachpixel. In some examples, such one or more circuits may include a chip onglass (“COG”) component 622. The COG component 622 may be disposed upona layer of glass in the assembly 603, such as the inner glass layer620B. In some implementations, the COG component 622 may be laterallyoffset from the array of pixels (e.g., array of pixel electrodes of theinner electrode layer 616B). In this way, the COG component 622 of theassembly 603 may be positioned outside of a user's field of view(“FOV”), obscured by housing or other components of the see-throughdisplay system, or a combination thereof. Pixels that use activeswitching elements such as TFTs are referred to as being activelyaddressed.

FIG. 7A illustrates an example of a see-through display system 700 in afirst state (state “A”). Similarly, FIG. 7B illustrates the see-throughdisplay system 700 in a second state (state “B”) that is different fromthe first state (state “A”). As shown in FIGS. 7A-7B, the system 700includes an eyepiece 702 and an optically-transmissive SLM or displayassembly 703. In some examples, the eyepiece 702 of FIGS. 7A-7Bcorresponds to one or more of components 202, 302A, 302B, and 502 asdescribed above with reference to FIGS. 2A-2C, 3, and 5, respectively.In some implementations, the assembly 703 of FIGS. 7A-7B may correspondto one or more of components 203, 303 a, 303B, 503, and 603 as describedabove with reference to FIGS. 2A-2C, 3, 5, and 6A-6B respectively.

The eyepiece 702 of the system 700 includes three waveguides 1210, 1220,and 1230. Each of the three waveguides 1210, 1220, and 1230 may, forexample, correspond to a different color of light and/or depth ofvirtual content. As shown in FIGS. 7A-7B, the eyepiece 702 furtherincludes incoupling optical elements 1212, 1222, and 1232 disposed uponwaveguides 1210, 1220, and 1230, respectively. The incoupling opticalelements 1212, 1222, and 1232 may be configured to couple light intowaveguides 1210, 1220, and 1230, respectively, for propagation via totalinternal reflection (TIR). In addition, the eyepiece 702 also includesoutcoupling diffractive optical elements 1214, 1224, and 1234 disposedupon waveguides 1210, 1220, and 1230, respectively. The outcouplingdiffractive optical elements 1214, 1224, and 1234 may be configured tocouple light out of waveguides 1210, 1220, and 1230, respectively,toward one or both eyes of a viewer of the system 700. As can be seen inFIG. 7A, in the first state (state “A”), no light is coupled into or outof the eyepiece 702.

In the example of FIG. 7B, the system 700 is in a second, differentstate (state “B”) in which light is (i) coupled into the waveguides1210, 1220, and 1230 by way of the incoupling optical elements 1212,1222, and 1232, (ii) propagating through the waveguides 1210, 1220, and1230 via total internal reflection (TIR), and (iii) coupled out of thewaveguides 1210, 1220, and 1230 by way of the outcoupling diffractiveoptical elements 1214, 1224, and 1234. As each light path 1240, 1242 and1244 respectively incouples at locations 1212, 1222, and 1232 impact arespective outcoupling diffractive optical element 1214, 1224, or 1234(outcoupled light from paths 1222 and 1232 not depicted) disposed uponwaveguide 1210, 1220, and 1230, it diffracts light both towards theviewer and away from the viewer, towards the world side of the device.This light propagation is depicted as multiple beamlets each propagatingin two opposite directions: one towards the viewer represented by lightbundle 3010, and one in a direction away from the viewer represented bylight bundle 3020.

The light bundle 3020 propagating away from the viewer may causeundesirable effects if it reflects off elements of the system in itspath because these reflections can reach the viewer as stray light. Forexample, reflections of this light from the subsequent waveguide 1220can interfere with light bundle 3010, increasing blurriness and/orreducing contrast of imagery projected by system 700. Furthermore, insome situations, the light bundle 3020 may reflect off of one or morecomponents of the assembly 703, such as one or more metal line traces orconductors of the assembly 703 that are equivalent or similar to themetal line traces or conductors 617 a-n of the assembly 603 as describedabove with reference to FIGS. 6A-6B, which may produce “ghost” imagesand other undesirable artifacts. Waveguide optical systems that employpupil expander technology can further aggravate these problems.

FIGS. 8A and 8B respectively illustrate an example of a see-throughdisplay system 800 in a first state (state “A”) and a second state(state “B”) that is different from the first state (state “A”). As shownin FIGS. 8A-8B, the system 800 includes eyepiece 702 and anoptically-transmissive SLM 803. In some examples, the eyepiece 702 ofFIGS. 8A-8B may correspond to the eyepiece 702 of FIGS. 7A-7B, but mayfurther include an outer glass cover 1250A and an inner glass cover1250B positioned on either side of the waveguide stack.

In some implementations, the assembly 803 of FIGS. 8A-8B may correspondto one or more of assemblies 603 and 703 as described above withreference to FIGS. 6A-6B and 7A-7B, respectively, but where the innerand outer optical retarders 614A and 614B are quarter wave plates (QWP)624A and 624B, respectively. The outer and inner QWPs 624A, 624B serveto convert linearly polarized light passing therethrough into circularlypolarized light. For example, light propagating from the viewer'senvironment toward the viewer can be linearly polarized as it passesthrough the outer polarizer 612A, and subsequently become circularlypolarized as it passes through the outer QWP 624A. Similarly, lightpropagating away from the viewer and toward the assembly 803, such asone or more portions of light bundle 3020, may become linearly polarizedas it passes through the inner polarizer 612B, and subsequently becomecircularly polarized as it passes through the inner QWP 624B. As can beseen in FIG. 8A, in the first state (state “A”), no light is coupledinto or out of the eyepiece 702.

In the example of FIG. 8B, the system 800 is in a second, differentstate (state “B”) in which light is coupled out of the eyepiece 702 intwo directions: one towards the viewer represented by light ray 3011,and one in a direction away from the viewer represented by light ray3021. Light rays 3011 and 3021 may represent portions of light bundlessimilar to light bundles 3010 and 3020 as described above with referenceto FIG. 7B. As shown in FIG. 8B, as the light ray 3021 propagates awayfrom the eyepiece 702 and toward the assembly 803, the light ray 3021 islinearly polarized as it passes through the inner polarizer 612B, andsubsequently become circularly polarized as it passes through the innerQWP 624B. Upon reaching one or more metal line traces or conductors ofthe assembly 803, the light ray 3021 may reflect back toward theeyepiece 702, which may effectively reverse the handedness of thecircular polarization of the light ray 3021 (i.e., left-handedcircularly polarized light becomes right-handed circularly polarizedlight up reflection, and vice versa). Passing through the inner QWP 624Bon its way back toward the eyepiece 702, the circularly polarized lightis converted back to linearly polarized light, but polarized orthogonalto the pass state of polarizer 612B. Accordingly, the reflected light isattenuated (e.g., blocked) by polarizer 612B. In this way, the system800 reduces the effects of “ghost” images that may be produced as aresult of light from the eyepiece 702 reflecting off of components ofthe assembly 803 and back toward the viewer.

In some embodiments, the inner optical retarder 614A is a quarter waveplate and the outer optical retarder 614B is an A-plate (e.g., auniaxial birefringent film with fast axis in the plane of the plate)with a retardation that is slightly different from quarter waveretardation. For example, in some implementations, the liquid crystalcan retain some residual birefringence even in its state of lowestbirefringence. This can occur, for instance, due to the orientation ofliquid crystal molecules close to alignment layers within a liquidcrystal layer. At and close to the alignment layer, the liquid crystalmolecules can retain their alignment even in the presence of maximumelectric field strength applied by the electrode structure. It ispossible to compensate for such residual retardation of the liquidcrystal material by increasing a retardation of the outer opticalretarder 614B above quarter wave retardation. For example, the outeroptical retarder 614B can be an A-plate with a retardation that is 5 nmor more greater than quarter wave retardation (e.g., 10 nm or more, 15nm or more, 20 nm or more, 25 nm or more, 30 nm or more, 35 nm or more,40 nm or more, 45 nm or more, such as up to 50 nm). Compensating forresidual retardation of the liquid crystal layer can reduce lightleakage through the dynamic dimmer.

The fast axes of the inner and outer optical retarders are generallyarranged relative to the alignment directions of the liquid crystallayer and the pass axes of the inner and outer polarizers so that thedynamic dimmer provides a large dynamic range with good light extinctionin its darkest state. In some embodiments, the inner and outer opticalretarders have their fast axes oriented at 45° relative to the pass axesof their adjacent polarizer. The fast axes of the inner and outeroptical retarders can be oriented at 90° with respect to each other. Incertain embodiments, the fast axis of the outer optical retarder isoriented at 90° with respect to an alignment direction of the liquidcrystal layer at the world-side boundary of the liquid crystal layer.

Generally, the performance of the dynamic dimmer is optimized for atleast one light propagation direction at at least one wavelength. Forexample, the dynamic dimmer's performance can be optimized for “on-axis”light, i.e., light that is incident normal to the layers of the stackforming the dynamic dimmer. In certain embodiments, the performance ofthe dynamic dimmer is optimized for green light, e.g., light with awavelength of 550 nm. Furthermore, while the performance of the dynamicdimmer may be optimal for one wavelength along one light propagationdirection, in general, the dimmer will be configured to provide adequateperformance over a range of angles and a range of wavelengths. Forexample, the dynamic dimmer can be configured to provide adequateperformance (e.g., a dynamic range above a minimum performancethreshold) over a range of light propagation angles the same size orlarger than the field of view of the display. Furthermore, theperformance can be adequate over a range of operative wavelengths (e.g.,spanning the color gamut of the display). To achieve adequateperformance over a range of wavelengths, in some implementations,achromatic optical components, such as achromatic optical retarders, canbe used. Achromatic A-plates (e.g., achromatic quarter wave plates) canbe provided by using two or more different birefringent materials thathave different dispersions, for instance.

FIG. 9 illustrates a further example of a see-through display system900. The system 900 of FIG. 9 may correspond to the system 800 of FIGS.8A-8B, but may further include an outer anti-reflective layer 626A andan inner anti-reflective layer 626B. The outer anti-reflective layer626A may be positioned adjacent the outer polarizer 612A, while theinner anti-reflective layer 626B may be positioned adjacent the innerpolarizer 612B. The outer and inner anti-reflective layers 626A, 626Bmay serve to further reduce the presence undesirable effects, such as“ghost” images, by limiting the reflection of light from coupled out ofthe eyepiece 702. In some situations, at least some such reflections maybe a result of refractive index mismatches between different layers ofdisplay system 900 or between air and a layer of display system 900(i.e., Fresnel reflections). Anti-reflective layers can reduce suchreflections. A variety of different appropriate anti-reflective layerscan be used, including single layer or multi-layer anti-reflectivefilms. Such layers can be optimized for on axis light or for light atsome other non-normal angle of incidence. Such layers can be optimizedfor one or more visible wavelengths. In some embodiments, theanti-reflective layers are optimized for light having a wavelength inthe green portion of the visible spectrum, e.g., at or near the maximumphotopic sensitivity of the human eye. Examples of anti-reflectivelayers for use in eyepieces are described in U.S. patent applicationSer. No. 16/214,575, filed on Dec. 10, 2018, published on Jun. 13, 2019as U.S. Publication No. 2019/0179057, which is expressly incorporated byreference herein in its entirety.

FIG. 10 illustrates yet another example of a see-through display system1000. The system 1000 of FIG. 10 may correspond to the system 900 ofFIG. 9, but may further include a second outer QWP 634A and a secondinner QWP 634B. The second outer QWP 634A is positioned adjacent theouter polarizer 612A, while the second inner QWP 634B is positionedadjacent the inner polarizer 612B. The second QWP 634A may serve tofurther reduce the presence undesirable effects, such as “ghost” images,by modulating the polarization state of light traveling away from theviewer and having passed through the outer polarizer 612A, such thatsaid light is attenuated by the outer polarizer 612A in the event thatit reflects off of one or more objects in the vicinity of the system1000 (e.g., beyond the assembly 803). Similarly, the inner QWP 634B mayserve to further reduce the presence undesirable effects, such as“ghost” images, by modulating the polarization state of light travelingtoward the viewer and having passed through the inner polarizer 612B,such that said light is attenuated by the inner polarizer 612A in theevent that it reflects off of one or more components of the system 1000positioned between the inner polarizer 612B and the viewer.

FIG. 11 illustrates another example of a see-through display system1100. The system 1100 of FIG. 11 may correspond to the system 1000 ofFIG. 10, but may include an index-matched layer 628 in place ofcomponent 626B. The index-matched layer 628 may be a layer of material(e.g., a gel or optical adhesive) having a refractive index equivalentor similar to the refractive index of the second inner QWP 634B orwhatever layer of assembly 803 is closest to eyepiece 702. Including anindex matching layer may serve to further reduce the presenceundesirable effects, such as “ghost” images, by reducing Fresnelreflections of light from eyepiece 702 incident on assembly 803.

In some implementations, a see-through display system may include someor all of the components from each of one or more of FIGS. 8A-11. Otherconfigurations, including see-through display systems with differentcombinations of components from each of one or more of FIGS. 8A-11, arepossible. Although described primarily within the context ofoptically-transmissive spatial light modulators and displays it is to beunderstood that one or more of the configurations and techniquesdescribed herein may be leveraged in other systems with see-throughpixel arrays.

Some implementations described in this specification can be implementedas one or more groups or modules of digital electronic circuitry,computer software, firmware, or hardware, or in combinations of one ormore of them. Although different modules can be used, each module neednot be distinct, and multiple modules can be implemented on the samedigital electronic circuitry, computer software, firmware, or hardware,or combination thereof.

Some implementations described in this specification can be implementedas one or more computer programs, i.e., one or more modules of computerprogram instructions, encoded on computer storage medium for executionby, or to control the operation of, data processing apparatus. Acomputer storage medium can be, or can be included in, acomputer-readable storage device, a computer-readable storage substrate,a random or serial access memory array or device, or a combination ofone or more of them. Moreover, while a computer storage medium is not apropagated signal, a computer storage medium can be a source ordestination of computer program instructions encoded in an artificiallygenerated propagated signal. The computer storage medium can also be, orbe included in, one or more separate physical components or media (e.g.,multiple CDs, disks, or other storage devices).

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing. The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages. A computer program may, but need not, correspondto a file in a file system. A program can be stored in a portion of afile that holds other programs or data (e.g., one or more scripts storedin a markup language document), in a single file dedicated to theprogram in question, or in multiple coordinated files (e.g., files thatstore one or more modules, sub programs, or portions of code). Acomputer program can be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a communication network.

Some of the processes and logic flows described in this specificationcan be performed by one or more programmable processors executing one ormore computer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andprocessors of any kind of digital computer. Generally, a processor willreceive instructions and data from a read only memory or a random accessmemory or both. A computer includes a processor for performing actionsin accordance with instructions and one or more memory devices forstoring instructions and data. A computer may also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto optical disks, or optical disks. However, a computer need nothave such devices. Devices suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices (e.g., EPROM, EEPROM, flash memory devices, and others),magnetic disks (e.g., internal hard disks, removable disks, and others),magneto optical disks, and CD ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in, special purposelogic circuitry.

To provide for interaction with a user, operations can be implemented ona computer having a display device (e.g., a monitor, or another type ofdisplay device) for displaying information to the user and a keyboardand a pointing device (e.g., a mouse, a trackball, a tablet, a touchsensitive screen, or another type of pointing device) by which the usercan provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well; for example, feedbackprovided to the user can be any form of sensory feedback, e.g., visualfeedback, auditory feedback, or tactile feedback; and input from theuser can be received in any form, including acoustic, speech, or tactileinput. In addition, a computer can interact with a user by sendingdocuments to and receiving documents from a device that is used by theuser; for example, by sending web pages to a web browser on a user'sclient device in response to requests received from the web browser.

A computer system may include a single computing device, or multiplecomputers that operate in proximity or generally remote from each otherand typically interact through a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), an inter-network (e.g., the Internet), a networkcomprising a satellite link, and peer-to-peer networks (e.g., ad hocpeer-to-peer networks). A relationship of client and server may arise byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

FIG. 12 shows an example computer system 1200 that includes a processor1210, a memory 1220, a storage device 1230 and an input/output device1240. Each of the components 1210, 1220, 1230 and 1240 can beinterconnected, for example, by a system bus 1250. The processor 1210 iscapable of processing instructions for execution within the system 1200.In some implementations, the processor 1210 is a single-threadedprocessor, a multi-threaded processor, or another type of processor. Theprocessor 1210 is capable of processing instructions stored in thememory 1220 or on the storage device 1230. The memory 1220 and thestorage device 1230 can store information within the system 1200.

The input/output device 1240 provides input/output operations for thesystem 1200. In some implementations, the input/output device 1240 caninclude one or more of a network interface device, e.g., an Ethernetcard, a serial communication device, e.g., an RS-232 port, and/or awireless interface device, e.g., an 802.11 card, a 3G wireless modem, a4G wireless modem, etc. In some implementations, the input/output devicecan include driver devices configured to receive input data and sendoutput data to other input/output devices, e.g., wearable display system1260. In some implementations, mobile computing devices, mobilecommunication devices, and other devices can be used.

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of exemplary configurations including implementations.However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques. Various changes maybe made in the function and arrangement of elements without departingfrom the spirit or scope of the disclosure.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the technology.Also, a number of steps may be undertaken before, during, or after theabove elements are considered. Accordingly, the above description doesnot bind the scope of the claims.

As used herein and in the appended claims, the singular forms “a”, “an”,and “the” include plural references unless the context clearly dictatesotherwise. Thus, for example, reference to “a user” includes a pluralityof such users, and reference to “the processor” includes reference toone or more processors and equivalents thereof known to those skilled inthe art, and so forth.

Also, the words “comprise”, “comprising”, “contains”, “containing”,“include”, “including”, and “includes”, when used in this specificationand in the following claims, are intended to specify the presence ofstated features, integers, components, or steps, but they do notpreclude the presence or addition of one or more other features,integers, components, steps, acts, or groups.

It is also understood that the examples and embodiments described hereinare for illustrative purposes only and that various modifications orchanges in light thereof will be suggested to persons skilled in the artand are to be included within the spirit and purview of this applicationand scope of the appended claims.

What is claimed is:
 1. A head-mounted apparatus, comprising: an eyepiececomprising a variable dimming assembly; and a frame mounting theeyepiece so that, during use of the head-mounted apparatus, a user sideof the eyepiece faces a towards a user of the head-mounted apparatus,and a world side of the eyepiece opposite the first side faces away fromthe user, wherein the dynamic dimming assembly is configured toselectively modulate an intensity of light transmitted parallel to anoptical axis from the world side of the eyepiece to the user side of theeyepiece during operation of the head-mounted apparatus, the dynamicdimming assembly comprising: a variable birefringence cell comprising aplurality of pixels each having an independently variable birefringence;a first linear polarizer arranged on the user side of the variablebirefringence cell, the first linear polarizer being configured totransmit light propagating parallel to the optical axis linearlypolarized along a pass axis of the first linear polarizer orthogonal tothe optical axis; a quarter wave plate arranged between the variablebirefringence cell and the first linear polarizer, a fast axis of thequarter wave plate being arranged relative to the pass axis of the firstlinear polarizer to transform linearly polarized light transmitted bythe first linear polarizer into circularly polarized light; and a secondlinear polarizer on the world side of the variable birefringence cell.2. The head-mounted apparatus of claim 1, wherein the dynamic dimmingassembly further comprises an optical retarder arranged between thevariable birefringence cell and the second linear polarizer.
 3. Thehead-mounted apparatus of claim 2, wherein the optical retarder is asecond quarter wave plate.
 4. The head-mounted apparatus of claim 2,wherein the optical retarder is an A-plate with a retardation greaterthan a retardation of the quarter wave plate.
 5. The head-mountedapparatus of claim 2, wherein a difference between a retardation of theoptical retarder and a retardation of the quarter wave plate correspondsto a residual retardation of the variable birefringent cell in a minimumbirefringence state.
 6. The head-mounted apparatus of claim 1, whereinthe variable birefringence cell comprises a layer of a liquid crystal.7. The head-mounted apparatus of claim 6, wherein the liquid crystal isconfigured in an electrically controllable birefringence mode.
 8. Thehead-mounted display of claim 6, wherein the layer of the liquid crystalis a vertically aligned liquid crystal layer.
 9. The head-mounteddisplay of claim 6, wherein the liquid crystal is a nematic phase liquidcrystal.
 10. The head-mounted display of claim 1, wherein the pixels ofthe variable birefringence cell are actively addressed pixels.
 11. Thehead-mounted apparatus of claim 1, wherein the eyepiece furthercomprises a see-through display mounted in the frame on the user side ofthe variable dimming assembly.
 12. The head-mounted apparatus of claim11, wherein the see-through display comprises one or more waveguidelayers arranged to receive light from a light projector during operationof the head-mounted apparatus and direct the light toward the user. 13.The head-mounted apparatus of claim 12, further comprising one or moreindex-matching layers arranged between the see-through display and thevariable dimming assembly.
 14. The head-mounted apparatus of claim 1,wherein the dynamic dimming assembly comprises one or moreantireflection layers.
 15. An augmented reality system comprising thehead-mounted apparatus of claim
 1. 16. A head-mounted apparatus,comprising: an eyepiece comprising a variable dimming assembly; and aframe mounting the eyepiece so that, during use of the head-mountedapparatus, a user side of the eyepiece faces a towards a user of thehead-mounted apparatus, and a world side of the eyepiece opposite thefirst side faces away from the user, wherein the dynamic dimmingassembly is configured to selectively modulate an intensity of lighttransmitted parallel to an optical axis from the world side of theeyepiece to the user side of the eyepiece during operation of thehead-mounted apparatus, the dynamic dimming assembly comprising: a layerof a liquid crystal; a circular polarizer arranged on the user side ofthe liquid crystal layer; and a linear polarizer on the world side ofthe liquid crystal layer.
 17. The head-mounted apparatus of claim 16,wherein the circular polarizer comprises a linear polarizer and aquarter wave plate.
 18. The head-mounted apparatus of claim 17, furthercomprising an A-plate arranged between the linear polarizer on the worldside of the liquid crystal layer.
 19. The head-mounted apparatus ofclaim 16, comprising a pixelated cell comprising the layer of the liquidcrystal, the pixelated cell being an actively addressed pixelated cell.